Disturbance detection in video communications

ABSTRACT

Embodiments disclosed herein provide systems, methods, and computer-readable media for detecting disturbances in a media stream from a participant on a communication. In a particular embodiment, a method provides receiving biometric information indicating a motion of the participant and determining that the motion indicates a visual disturbance in a video component of the first media stream. The method further provides identifying the visual disturbance in the video component of the first media stream and removing the visual disturbance from the video component of the first media stream.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/823,088, entitled “DISTURBANCE DETECTION IN VIDEO COMMUNICATIONS,”filed Aug. 11, 2015, which is hereby incorporated by reference in itsentirety.

TECHNICAL FIELD

Aspects of the disclosure are related to video communications betweenparticipants and, in particular, to detecting disturbances in thosevideo communications.

TECHNICAL BACKGROUND

Video is now a popular form of communicating between two or moreparties. Both audio and video is captured of participants on a videocommunication and transferred to the other participants. Like audiocommunications, video communications can be established between twoparties or between more than two parties, as may be the case during avideo conference. The video component of such communications improvesupon audio communications in that a participant on the communication isable to see other participants along with other visual aids that thoseparticipants may be presenting. At the very least, the facialexpressions that can be seen in the video communication can add moredepth to the conversation itself that might otherwise have been missedover an audio only connection.

Conversely, video communications are more prone to disturbances sinceparticipants not only hear other participants but they also see them.Thus, each participant that is captured on video must be aware of theirappearance in that video. For example, a participant that coughs duringan audio-only communication would simply have that cough heard by otherparticipants. However, when video accompanies that audio, theparticipant may need to cover their mouth or turn away from the videocamera in conjunction with the cough. Such actions would not be neededif the participant was not captured on video during the cough. Likewise,other conversational norms may need to be followed during a videocommunication (e.g. eye contact, smiling, etc.) that would not otherwiseneed to be followed to maintain a disturbance-free audio communication.

OVERVIEW

Embodiments disclosed herein provide systems, methods, andcomputer-readable media for detecting disturbances in a media streamfrom a participant on a communication. In a particular embodiment, amethod provides receiving biometric information indicating a motion ofthe participant and determining that the motion indicates a visualdisturbance in a video component of the first media stream. The methodfurther provides identifying the visual disturbance in the videocomponent of the first media stream and removing the visual disturbancefrom the video component of the first media stream.

In some embodiments, the method includes identifying an audibledisturbance in an audio component of the first media stream and usingthe visual disturbance to confirm the audible disturbance. In thoseembodiments, the method may include removing the audible disturbancefrom the audio component of the first media stream. Also in thoseembodiments, the method may include replacing the audible disturbancewith background noise in the audio component of the first media stream.

In some embodiments, removing the visual disturbance includes replacingthe visual disturbance with prerecorded frames of the participant thatdo not include a disturbance. In those embodiments, the method mayinclude performing visual processing on the video component of the firstmedia stream to ensure a transition to the prerecorded frames does notcreate a disturbance.

In some embodiments, the method includes determining a disturbance scorefor the participant based on the visual disturbance. In thoseembodiments, the method may include determining the disturbance scorebased on an amount of disturbances and a frequency of disturbances.

In some embodiments, the method includes providing the participant withfeedback concerning the visual disturbance. In those embodiments, thefeedback may include at least one recommendation for mitigating visualdisturbances.

In another embodiment, a system is provided having one or more computerreadable storage media and a processing system operatively coupled withthe one or more computer readable storage media. Program instructionsstored on the one or more computer readable storage media, when read andexecuted by the processing system, direct the processing system toreceive biometric information indicating a motion of the participant anddetermine that the motion indicates a visual disturbance in a videocomponent of the first media stream. The program instructions furtherdirect the processing system to identify the visual disturbance in thevideo component of the first media stream and remove the visualdisturbance from the video component of the first media stream.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. While several implementations are describedin connection with these drawings, the disclosure is not limited to theimplementations disclosed herein. On the contrary, the intent is tocover all alternatives, modifications, and equivalents.

FIG. 1 illustrates a video communication environment for detectingdisturbances in video communications.

FIG. 2 illustrates a method of operating the video communicationenvironment to detect disturbances in video communications.

FIG. 3 illustrates a video communication environment for detectingdisturbances in video communications.

FIG. 4 illustrates an operational scenario of the video communicationenvironment to detect disturbances in video communications.

FIG. 5 illustrates an operational scenario of the video communicationenvironment to detect and correct disturbances in video communications.

FIG. 6 illustrates an operational scenario of the video communicationenvironment to detect disturbances in video communications.

FIG. 7 illustrates an exemplary media stream segment when detectingdisturbances in video communications.

FIG. 8 illustrates a disturbance system for detecting disturbances invideo communications.

TECHNICAL DISCLOSURE

During a video communication, disturbances caused by participants on thecommunication are captured just as though the participants are speakingin person. These disturbances may include coughs, sneezes, lack of eyecontact, sniffles, subconscious movements, or any other action that maydetract from the conversation. While these disturbances may be forgivenin some communications, such as a video call between friends, in othersituations the disturbances may have a greater effect. For example, if avideo communication is part of a contact center communication between acustomer and an agent, then the customer may be adversely effected bythe amount and type of disturbances during the video communication (e.g.make a less than favorable decision due to the disturbances by theagent). In a similar example, when a participant is a presenter on avideo conference, the disturbances that the presenter creates mayadversely effect the other participants' assimilation of information.

Accordingly, the disturbance system described herein detects thedisturbances in a media stream captured of at least one participant on avideo communication. Based on the detected disturbances, actions can betaken to mitigate the disturbances either during the communication beinganalyzed for disturbances or at some future time. For example, theparticipant may be notified of the disturbances or trained to avoidcausing the disturbances in the future. In other examples, thedisturbance system may correct the media stream to mitigate thedisturbance in real time during the communication. Regardless of whatfeatures are implemented, the disturbance system is meant to reduce theamount of disturbances in video communications in order to improve thevideo communication experience.

FIG. 1 illustrates video communication environment 100 in an example ofdetecting disturbances in video communications. Video communicationenvironment 100 includes disturbance system 101, video client system102, and video client system 103. Video client systems 102 and 103 areoperated by participants 122 and 123, respectively. Video client system102 and disturbance system 101 communicate over communication link 111.Video client system 103 and disturbance system 101 communicate overcommunication link 112.

In this example, disturbance system 101 is shown on the communicationpath between video client systems 102 and 103. However, disturbancesystem 101 may be located on a different communication path.Additionally, in some examples, the functionality of disturbance system101 may be incorporated into one of video client systems 102 and 103.Alternatively, disturbance system 101 may be incorporated into a videoservice system (e.g. video conference system) that facilitates videocommunications between video client systems 102 and 103.

FIG. 2 illustrates a method 200 for operating video communicationenvironment 100 to detect disturbances in video communications. Method200 provides disturbance system 101 identifying disturbance criteriadefining audible disturbances, visual disturbances, and communicationdisturbances (step 201). The various disturbances may be defined basedon definition information provided by an administrator of disturbancesystem 101, may be preprogrammed into disturbance system 101, may belearned from previous video communications, or from some other source.The disturbance definitions may apply to specific participants or mayapply to multiple participants. In one example, a disturbance may begenerally defined, such as a participant touching their face with theirhand. Video image processing will then be relied upon to determinewhether the participant performs that face touching action. In anotherexample, the disturbance may be defined more specifically. For instance,instead of merely defining a cough as being a disturbance, a waveformmay be provided that represents a cough for use in comparison. Otherways of defining a disturbance may also be used depending on the mediaprocessing capabilities of disturbance system 101.

The disturbances defined by the disturbance criteria may includephysiological events, such as yawns, sneezes, coughs, hiccups, sniffles,and the like. Likewise, the disturbances may include non-speechutterances, such as throat clears, clicks, cries, grunts, laughs, noise,pauses, sighs, smacks, swallows, and the like. The disturbances mayfurther include emotional expressions such as frowns, grimaces, or othertypes of unpleasant expressions. Additionally, facial contact ormannerisms, such as brushing hair, straightening glasses, touching,scratching, or rubbing of the face, and the like, may also be defined asdisturbances. Other or different types of disturbances may also bedefined and may vary from participant to participant.

Method 200 then provides identifying one or more audible disturbancesfrom an audio component of a media stream from video client system 102to video client system 103 based on predefined disturbance criteria(step 202). The media stream comprises data communications representingthe audio and video captured of participant 122 for a videocommunication between participant 122 and participant 123 using theirrespective video client systems 102 and 103. The video communication maybe a one-on-one communication between participants 122 and 123 or,though not shown, may include additional participants and video clientsystems. In some examples, a video communication system may be used tofacilitate the video communication between video client system 102 andvideo client system 103. Additionally, the video communication may useany video communication protocol, such as WebRTC, Real-Time Protocol(RTP), or some other protocol. While the media stream referred to inthis example is from video client system 102, method 200 may beperformed on the media stream from video client system 103 or any othervideo client system on the video communication.

When identifying the audible disturbances, disturbance system 101 mayneed to decode the audio component of the media stream. Disturbancesystem 101 then analyzes the audio in the audio component to determinewhether any of the audio qualifies as an audible disturbance based onthe disturbance criteria. In some examples, disturbance system 101 maycompare the audio waveform to waveforms in the disturbance criteria todetermine whether a portion of the audio waveform substantially matchesa waveform in the criteria. If so, then disturbance system 101 willidentify that portion of the audio waveform as a disturbance of the typecorresponding to the matched waveform in the criteria. Additionally,speech recognition techniques that commonly model laughter, coughs,sneezes, and the like, may also be used to identify audibledisturbances.

In addition to audible disturbances, method 200 provides identifyingvisual disturbances from a video component of the media stream based onthe disturbance criteria (step 203). The visual disturbances may beidentified using image processing analysis. The image processinganalysis may determine participant 122's head positioning, eyepositioning, head movement, obscuring of portions of the head (e.g. withparticipant 122's hand or otherwise), or any other type of visual cuethat can be identified from a video image. The video component may becaptured by one or more camera components of video client system 102 andmay be positioned in multiple locations at participant 122's location.The cameras may include video cameras, video camera arrays, plenopticcameras, depth sensors, and 3D cameras—including combinations thereof.

Method 200 also provides correlating the audible disturbances with thevisual disturbances to determine one or more combined disturbances forparticipant 122 based on the disturbance criteria (step 204). Each ofthe combined disturbances comprises at least one of the audibledisturbances and at least one of the visual disturbances, which occursubstantially contemporaneously so as to be considered components of thesame disturbance. For example, while lack of eye contact may be definedas a visual disturbance, lack of eye contact has no audio component thatwould result in the identification of a combined disturbance. However,something like a sneeze will typically make a sound and involve headmovement of some sort (e.g. closing of the eyes or tilting the head).Thus, a disturbance like a sneeze will be identified as a combineddisturbance having both an audible and a visual component. In analternative example, a cough may only be defined as a disturbance ifparticipant 122 does not cover their mouth when coughing. Thus, for acough to be considered a combined disturbance in that example, both anaudible disturbance having the sound of a cough would need to occurcontemporaneously with a visual disturbance showing a coughing headmotion without a hand covering participant 122's face.

As alluded to in the cough example above, a disturbance may beidentified as the absence of some element, with the element in that casebeing the covering of participant 122's mouth during the cough. Thedisturbance criteria may therefore include definitions of absences thatwould constitute an audible, visual, or combined disturbance. In thecough example, the disturbance criteria may indicate that the absence ofan “excuse me” or the like from participant 122 following the cough isan audible disturbance. In another example, the absence of blinkingand/or eye movement may indicate that participant 122 is staring whichthe disturbance criteria may define as a visual disturbance.

In some cases, correlating the audible and visual disturbances mayidentify false positives or false negatives. That is, a visualdisturbance may be negated by having no corresponding audibledisturbance and vice versa. For example, an audible disturbance may beidentified as a sneeze while no corresponding visual disturbance havinghead/eye movement consistent with a sneeze was identified. Thus, theaudible disturbance may be marked as a false positive. Alternatively, avisual disturbance may be consistent with a sneeze while no audibledisturbance consistent with a sneeze was identified, as may be the caseif participant 122 has an atypical sounding sneeze. In that instance,the audio during the sneeze may be marked as a false negative. Thesefalse positives and false negatives may be used to fine tune thedisturbance criteria for future disturbance identifications.

Once identified, disturbance system 101 may store (either locally or ina separate storage system) the audio and video components of each of theaudible, visual, and combined disturbances, may store informationdescribing each disturbance (e.g. type, duration, etc.), may store theentire media stream while indicating where in the media streamdisturbances occurred, or some other type of information relevant to thedisturbances—including combinations thereof. In some cases, a combineddisturbance may replace its audible and visual disturbance components infavor of the combined disturbance. In other cases, the audible and/orvisual disturbance will remain in addition to the combined disturbanceof which they are a part. Thus, the disturbances and/or disturbanceinformation may be processed or analyzed after the disturbances havebeen identified by disturbance system 101.

In particular, disturbance system 101 may process the disturbances todetermine a score for the participant based on the disturbances. Factorsthat affect the disturbance score may be the total amount ofdisturbances, the frequency of disturbances, the type of disturbance,whether participant 122 is speaking or listening to other participantsat the time of the disturbance, or some other factor. The type ofdisturbance may be whether the disturbance is audible, visual, orcombined, or may also be defined more specifically, such as physicalreflexes, expressions, non-verbal vocalizations, etc. The score may beon any scale having any granularity (e.g. 1-10, low/medium/high, etc.).In general, larger numbers of disturbances and higher frequencies ofdisturbances will be scored to indicate a worse performance byparticipant 122. However, different disturbances or different types ofdisturbances may be weighted differently depending on the severity ofdisturbance, which may be defined in the disturbance criteria.

Additionally, disturbance system 101 may remove or repair disturbancesin the media stream. For audible disturbances, disturbance system 101may remove the audio for the disturbance from the media stream. Theaudio may be replaced with background noise (possibly recorded fromprior in the communication) to ensure the silence is not as noticeableto participant 123 when hearing the audio from the media stream.Similarly, a visual disturbance may be replaced by pre-recorded video ofparticipant 122. The video may be recorded from earlier in thecommunication or may have been recorded at some other time. Ideally, thepre-recorded video segment meshes with participant 122's position in thevideo frame such that video continuity is substantially maintained forparticipant 123.

It should be understood that in method 200 the audible disturbances andthe visual disturbances need not be identified in the order shown. Steps202 and 203 may therefore be performed in any order orcontemporaneously. Likewise, if done in sequence, one of steps 202 or203 need not be complete before the other can begin. The identificationof combined disturbances at step 204 can occur concurrently with theidentification of the audible and visual disturbances or sometimethereafter.

Referring back to FIG. 1, disturbance system 101 comprises a computerprocessor system and communication interface. Disturbance system 101 mayalso include other components such as a router, server, data storagesystem, and power supply. Disturbance system 101 may reside in a singledevice or may be distributed across multiple devices. While shownseparately, disturbance system 101 may be incorporated into one or moreof video client systems 102 and 103. Disturbance system 101 may be avideo communication server, conferencing system, application server,personal computer workstation, network gateway system, or some othercomputing system—including combinations thereof.

Video client systems 102 and 103 each comprise a computer processorsystem, at least one video camera, at least one microphone, andcommunication interface. Video client systems 102 and 103 may alsoinclude other components such as a router, server, data storage system,and power supply. Video client systems 102 and 103 may each reside in asingle device or may be distributed across multiple devices.Alternatively, client 103 may be a more traditional videoconferencingclient without the view control functionality. Video client systems 102and 103 may be a telephone, computer, tablet, e-book, mobile Internetappliance, network interface card, media player, game console,application server, proxy server, or some other communicationapparatus—including combinations thereof.

Communication links 111-112 use metal, glass, air, space, or some othermaterial as the transport media. Communication links 111-112 could usevarious communication protocols, such as Time Division Multiplex (TDM),Internet Protocol (IP), Ethernet, communication signaling, Code DivisionMultiple Access (CDMA), Evolution Data Only (EVDO), WorldwideInteroperability for Microwave Access (WIMAX), Global System for MobileCommunication (GSM), Long Term Evolution (LTE), Wireless Fidelity(WIFI), High Speed Packet Access (HSPA), or some other communicationformat—including combinations thereof. Communication links 111-112 couldbe direct links or may include intermediate networks, systems, ordevices.

FIG. 3 illustrates video communication environment 300 in an example ofdetecting disturbances in video communications. Video communicationenvironment 300 includes disturbance system 301, video client system302, video client system 303, communication network 304, and one or morebiometric sensors 342. Video client systems 302 and 303 are operated byagent 322 and caller 323, respectively. Video client system 302 andcommunication network 304 communicate over communication link 311. Videoclient system 303 and communication network 304 communicate overcommunication link 312. Disturbance system 301 and communication network304 communicate over communication link 313.

Communication network 304 comprises network elements that providecommunications services. Communication network 304 may compriseswitches, wireless access nodes, Internet routers, network gateways,application servers, computer systems, communication links, or someother type of communication equipment—including combinations thereof.Communication network 304 may be a single network (e.g. local areanetwork, wide area network, the Internet, etc.) or may be a collectionof networks.

Disturbance system 301 is a system connected to communication network304 along with video client systems 302 and 303. Media streams forcommunications between video client systems 302 and 303 may pass throughdisturbance system 301 or disturbance system 301 may receive copies ofany media streams upon which it operates. While not shown disturbancesystem 301 may be incorporated into a system that facilitates the videocommunications between video client systems 302 and 303. Alternatively,disturbance system 301 may be incorporated into one or more of videoclient systems 302 and 303.

In this example, agent 322 is employed by a contact center and is taskedwith representing an entity associated with the contact center. Agent322 may be charged with selling products or services provided by theentity, providing technical support for products sold by the company, orserve any other purpose that can be performed with callers over a videocommunication. Likewise, caller 323 is a customer or potential customerthat may be dialing into the contact center for assistance from one ofthe contact center's agents on a video communication. The videocommunication may be routed to agent 322 based on agent 322's expertise,role within the contact center, availability, or for some other reason.It is in the contact center's and the entity's best interest for thevideo communication to proceed with minimal disturbances. Therefore, asdiscussed in more detail below, the contact center uses disturbancesystem 301 to identify disturbances to assist agents to minimizedisturbances (either on a current communication, at a later time throughtraining, or through the contact center as a whole through staffingdecisions based on disturbances), to remedy disturbances as they happen,or some combination thereof.

FIG. 4 illustrates an operational scenario 400 of video communicationenvironment 300 to detect disturbances in video communications. In thisexample, a video communication has been established between agent 322and caller 323. Accordingly, audio and video is captured of agent 322and caller 323 by their respective video client systems 302 and 303. Theaudio and video captured by each video client system is converted into amedia stream for transfer to the other video client system for display.This process continues in substantially real-time to allow for real-timevideo communications between agent 322 and caller 323.

From the video and audio of the media stream, disturbance system 301uses method 200 from above to identify audible, visual, and combineddisturbances. However, in this embodiment, disturbance system 301 alsoreceives biometric information about agent 322. The biometricinformation is captured by biometric sensors 342 positioned on or aroundagent 322. The biometric sensors may be built into another component ofvideo client system 302 or may be independent components of video clientsystem 302 that communicate using wired or wireless signaling (e.g.Bluetooth or WiFi). Biometric sensors 342 may include a heart ratemonitor, a blood pressure monitor, an accelerometer, or any other typeof sensor that can gather information about agent 322 to assist inidentifying disturbances. The biometric information may be correlatedalong with audible, visual, and combined disturbances to increase theefficacy of disturbance system 301 to identify disturbances. Likewise,the biometric information may allow for disturbances to be identifiedmore quickly than they otherwise would.

The disturbance criteria that disturbance system 301 uses to identifydisturbances may further define biometric information that wouldindicate a disturbance. For example, the disturbance criteria mayinclude a head motion pattern that is typical of a sneeze. Thus, inaddition to an audible and/or visual disturbance that indicates asneeze, biometric information from an accelerometer on agent 322's headmay further indicate that agent 322's head moved in a manner consistentwith a sneeze. Moreover, the biometric information may be able to beinterpreted faster than image processing can identify the head movementfrom the video of agent 322.

Additionally, though not shown in this embodiment, video communicationenvironment 300 may include environmental sensors that captureinformation about agent 322's location, such as temperature, lighting,humidity, and the like. The disturbance criteria may therefore alsofactor in certain environmental conditions when defining disturbances.

Once audible, visual, and combined disturbances, if any, are identified,disturbance system 301 determines a disturbance score for agent 322. Thescore may be generated for the individual communication between agent322 and caller 323, for a portion of that communication, for multiplecommunications with agent 322 (e.g. within a period of time, such ashour, day, week, etc.), for a group of agents at the contact center, forthe contact center as a whole, or for some other delineation—includingcombinations thereof. Disturbance system 301 may adjust the score as thecommunication progresses or may wait until some time after thecommunication ends so that all the disturbances identified on thecommunication can be factored into the score together. The score maythen be used to determine which agents have issues with disturbances sothat proper actions (e.g. training, re-staffing, etc.) can be taken toimprove the score.

Additionally, in this embodiment, disturbance system 301 providesfeedback during or after the communication to video client system 302.Video client system 302 can then indicate the feedback visually,audibly, or in some other way, to agent 322. Upon getting the feedback,agent 322 can adjust their actions on the communication or in futurecommunications to reduce the number of disturbances and, if a score isgenerated, improve their disturbance score. For example, if disturbancesystem 301 has identified that the agent is touching their face too much(e.g. above a threshold frequency or number of times in thecommunication), agent 322 may receive feedback indicating that agent 322should stop touching their face.

In some embodiments, the media stream may not include a video component,as may be the case in traditional call center communications, when videois not supported by a caller, when video is not desired by a caller, orfor some other reason. In these cases, the biometric information may beused in a manner similar to that described above by disturbance system301 to identify audible disturbances without also identifying visual andcombined disturbances. For example, an accelerometer reading indicatinga head nod consistent with a sneeze may be correlated with audio of thesneeze to confirm that a sneeze has occurred. Alternatively, a visualcomponent may still be captured used by disturbance system 301 in themanner disclosed above to assist in identifying audible disturbanceseven though the video component of the media stream is not transferredto video client system 303. For example, head movement in the videocomponent that is consistent with a sneeze may be used to confirm thatan audible disturbance is a sneeze.

FIG. 5 illustrates another operational scenario 500 of videocommunication environment 300 to detect and correct disturbances invideo communications. In this example, a video communication has beenestablished in much the same way as the communication in operationalscenario 400. However, in this example, disturbance system 301 iscapable of correcting at least some of the audible, visual, or combineddisturbances. In order for correction to work, the disturbances must beidentified in real-time so that the disturbance identification and thecorrection do not delay and disturb the communication any more than thedisturbance itself.

When a disturbance is identified by disturbance system 301, disturbancesystem 301 determines whether the disturbance is one that can becorrected and, if so, what correction is needed. In some cases, audibledisturbances (or the audible component of a combined disturbance) may becorrected by removing them from the audio component of the media stream.Alternatively, if an audible disturbance cannot be removed in time tonot further disturb the communication, then a pre-recorded message fromagent may be added after the disturbance. For example, if agent 322coughs and does not make a statement akin to “excuse me” after thecough, disturbance system 301 may insert prerecorded audio of such astatement after the cough as a correction. Any correction applied to theaudio should not overlap any of agent 322's non-disturbance speech.Additionally, a visual disturbance (or the visual component of acombined disturbance) may be corrected by replacing with video framesshowing agent 322 without the disturbance. Those replacement frames maybe prerecorded or synthesized by disturbance system 301 or some othersystem. Some visual processing may be needed to ensure the transitionfrom the communication video frames to the replacement frames is not adisturbance in itself.

In real-time the corrections are made to the media stream before themedia stream is transferred to video client system 303. Thus, caller 323does not have to experience the disturbance when the media stream ispresented by video client system 303. For example, if agent 322 sneezes,then caller 323 will not see or hear the sneeze due to the correctionsapplied by disturbance system 301.

FIG. 6 shows another operational scenario 600 of video communicationenvironment 300 to detect disturbances in video communications. In thisexample, the functions of disturbance system 301 are incorporated intovideo client system 302. Thus, client system 302 captures audio andvideo of agent 322 through its camera and microphone interface. Theninstead of transferring the media stream to disturbance system 301,client system 302 identifies audible, visual, and combined disturbancesitself. In this example, video client system 302 further generates adisturbance score. However, in other examples, video client system 302may transfer information about the identified disturbances to anothersystem, such as disturbance system 301, to generate the score. Also,like in operational scenario 400, video client system 302 may providefeedback to agent 322 about the identified disturbances.

FIG. 7 illustrates an exemplary media stream segment 700 when detectingdisturbances in video communications. Audio segment 701 represents a 1.8second portion of the audio component of a media stream. Within thataudio segment is a speech segment 721, a cough segment 722, and anotherspeech segment 723. In the embodiments above, the 0.6 seconds thatcontain cough segment 722 would be identified as an audible disturbance.Cough segment 722 may then be factored into a disturbance score orinformation regarding cough segment 722 may be stored for futurereference. Moreover, in some examples, cough segment 722 is correctedwith silence segment 732. Silence segment 732 may be complete silence,may be a removal of the speaking voice while leaving background noise sothat the silence is not as noticeable, or may comprise some other meansof removing cough 722 from the audio. Additionally, though not shown, avideo segment corresponding to audio segment 701 may also be correctedto remove video frames showing the coughing action.

FIG. 8 illustrates disturbance system 800. Disturbance system 800 is anexample of disturbance system 101, although system 101 could usealternative configurations. Disturbance system 800 comprisescommunication interface 801, user interface 802, and processing system803. Processing system 803 is linked to communication interface 801 anduser interface 802. Processing system 803 includes processing circuitry805 and memory device 806 that stores operating software 807.Disturbance system 800 may include other well-known components such as abattery and enclosure that are not shown for clarity. Disturbance system800 may be a personal computer, tablet, application server, conferencingsystem, private branch exchange, or some other computingapparatus—including combinations thereof. In some examples, disturbancesystem 800 may be replicated across multiple locations in order todistribute the processing load required to function as described herein.

Communication interface 801 comprises components that communicate overcommunication links, such as network cards, ports, RF transceivers,processing circuitry and software, or some other communication devices.Communication interface 801 may be configured to communicate overmetallic, wireless, or optical links. Communication interface 801 may beconfigured to use TDM, IP, Ethernet, optical networking, wirelessprotocols, communication signaling, or some other communicationformat—including combinations thereof.

User interface 802 comprises components that interact with a user toreceive user inputs and to present media and/or information. Userinterface 802 may include a speaker, microphone, buttons, lights,display screen, touch screen, touch pad, scroll wheel, communicationport, or some other user input/output apparatus—including combinationsthereof. User interface 802 may be omitted in some examples.

Processing circuitry 805 comprises microprocessor and other circuitrythat retrieves and executes operating software 807 from memory device806. Memory device 806 comprises a non-transitory storage medium, suchas a disk drive, flash drive, data storage circuitry, or some othermemory apparatus. Processing circuitry 805 is typically mounted on acircuit board that may also hold memory device 806 and portions ofcommunication interface 801 and user interface 802. Operating software807 comprises computer programs, firmware, or some other form ofmachine-readable processing instructions. Operating software 807includes disturbance criteria module 808 and disturbance identificationmodule 809. Operating software 807 may further include an operatingsystem, utilities, drivers, network interfaces, applications, or someother type of software. When executed by processing circuitry 805,operating software 807 directs processing system 803 to operatedisturbance system 800 as described herein.

In particular, disturbance criteria module 808 directs processing system803 to identify disturbance criteria defining a plurality of audibledisturbances, a plurality of visual disturbances, and a plurality ofcommunication disturbances. Disturbance identification module 809directs processing system 803 to identify one or more audibledisturbances from an audio component of the media stream based onpredefined disturbance criteria and identify one or more visualdisturbances from a video component of the media stream based on thedisturbance criteria. Disturbance identification module 809 furtherdirects processing system 803 to correlate the audible disturbances withthe visual disturbances to determine one or more combined disturbancesfor the participant based on the disturbance criteria, wherein each ofthe combined disturbances comprises at least one of the audibledisturbances and at least one of the visual disturbances. The includeddescriptions and figures depict specific implementations to teach thoseskilled in the art how to make and use the best option. For the purposeof teaching inventive principles, some conventional aspects have beensimplified or omitted. Those skilled in the art will appreciatevariations from these implementations that fall within the scope of theinvention. Those skilled in the art will also appreciate that thefeatures described above can be combined in various ways to formmultiple implementations. As a result, the invention is not limited tothe specific implementations described above, but only by the claims andtheir equivalents.

What is claimed is:
 1. A method for detecting disturbances in a firstmedia stream captured of a participant on a communication, the methodcomprising: receiving biometric information indicating a motion of theparticipant, wherein the biometric information is captured by abiometric sensor of a video client system; determining that the motionindicates an audible disturbance in an audio component of the firstmedia stream, wherein the audio component is captured by a microphone ofthe video client system; and removing the audible disturbance from theaudio component.
 2. The method of claim 1, further comprising:identifying a visual disturbance in a video component of the first mediastream; and using the visual disturbance to confirm the audibledisturbance.
 3. The method of claim 2, further comprising: removing thevisual disturbance from the video component of the first media stream.4. The method of claim 1, wherein removing the audible disturbancecomprises: replacing the audible disturbance with background noise inthe audio component of the first media stream.
 5. The method of claim 3,wherein removing the visual disturbance comprises: replacing the visualdisturbance in the video component with prerecorded frames of theparticipant that do not include a disturbance.
 6. The method of claim 5,further comprising: performing visual processing on the video componentof the first media stream to ensure a transition to the prerecordedframes does not create a disturbance.
 7. The method of claim 1, furthercomprising: determining a disturbance score for the participant based onthe audible disturbance.
 8. The method of claim 7, further comprising:determining the disturbance score based on an amount of disturbances anda frequency of disturbances.
 9. The method of claim 1, furthercomprising: providing the participant with feedback concerning theaudible disturbance.
 10. The method of claim 9, wherein the feedbackincludes at least one recommendation for mitigating disturbances.
 11. Asystem for detecting disturbances in a first media stream captured of aparticipant on a communication, the system comprising: one or morecomputer readable storage media; a processing system operatively coupledwith the one or more computer readable storage media; and programinstructions stored on the one or more computer readable storage mediathat, when read and executed by the processing system, direct theprocessing system to: receive biometric information indicating a motionof the participant, wherein the biometric information is captured by abiometric sensor of a video client system; determine that the motionindicates an audible disturbance in an audio component of the firstmedia stream, wherein the audio component is captured by a microphone ofthe video client system; and remove the audible disturbance from theaudio component.
 12. The system of claim 11, wherein the programinstructions further direct the processing system to: identify a visualdisturbance in a video component of the first media stream; and use thevisual disturbance to confirm the audible disturbance.
 13. The system ofclaim 12, wherein the program instructions further direct the processingsystem to: remove the visual disturbance from the video component of thefirst media stream.
 14. The system of claim 11, wherein to remove theaudible disturbance, the program instructions direct the processingsystem to: replace the audible disturbance with background noise in theaudio component of the first media stream.
 15. The system of claim 13,wherein to remove the visual disturbance, the program instructionsdirect the processing system to: replace the visual disturbance in thevideo component with prerecorded frames of the participant that do notinclude a disturbance.
 16. The system of claim 15, wherein the programinstructions further direct the processing system to: perform visualprocessing on the video component of the first media stream to ensure atransition to the prerecorded frames does not create a disturbance. 17.The system of claim 11, wherein the program instructions further directthe processing system to: determine a disturbance score for theparticipant based on the audible disturbance.
 18. The system of claim17, wherein the program instructions further direct the processingsystem to: determine the disturbance score based on an amount ofdisturbances and a frequency of disturbances.
 19. The system of claim11, wherein the program instructions further direct the processingsystem to: provide the participant with feedback concerning the audibledisturbance.
 20. One or more non-transitory computer readable storagemedia, having instructions stored thereon for detecting disturbances ina first media stream captured of a participant on a communication, theinstructions, when read and executed by a processing system, direct theprocessing system to: receive biometric information indicating a motionof the participant, wherein the biometric information is captured by abiometric sensor of a video client system; determine that the motionindicates an audible disturbance in an audio component of the firstmedia stream, wherein the audio component is captured by a microphone ofthe video client system; and remove the audible disturbance from theaudio component.